When the Robot Makes a Judgment Call: MES for Physical AI Cells

Industrial robot arm working at an assembly line pilot cell

For forty years, the deal between MES and the robot on the floor has been simple. The robot runs a fixed program. The controller reports a completed cycle, a fault code, maybe a force curve if you’re lucky enough to have instrumented the tooling. MES logs that the step happened, stamps it to a serial number, and moves on. The robot doesn’t decide anything. It executes.

That deal is starting to break, and it’s breaking on pilot lines right now, not in a research lab. Through 2025 and into 2026, vendors like NVIDIA (with its Isaac platform and GR00T foundation model work), Figure, and a handful of other humanoid and manipulation-robotics companies have been pushing general-purpose robot policies — trained largely in simulation, fine-tuned on real demonstrations — into actual pilot cells doing actual assembly and material-handling tasks. These aren’t demo-day showpieces anymore. They’re bolted into real work cells, running real cycle time targets, next to real quality gates.

The engineering community has spent a lot of energy on whether these robots are ready. That’s a fair question and the honest answer is “for narrow, well-bounded tasks, increasingly yes.” But there’s a second question nobody’s forcing yet, and it’s the one that actually lands on an MES admin’s desk: what do you do with a robot that doesn’t run the same program twice?

The assumption MES has always made, and why it stops working

Traditional MES and quality architectures — think ISA-95’s execution layer, ISA-88 batch records, most genealogy schemas — assume determinism. A given work instruction plus a given recipe produces a given, predictable sequence of physical actions. If the robot deviated, that was a fault: an E-stop, a torque exceedance, a vision system rejecting a part. Deviation was binary and it was bad.

A foundation-model-driven manipulation policy doesn’t work that way. It’s making continuous, in-the-moment decisions based on visual and force feedback — how to approach a part that’s sitting slightly off-nominal, how to re-grip after a slip, which of several valid paths to take to complete an insertion. Sometimes that adaptability is exactly why you bought the thing: it handles variation a fixed program would have faulted on. But “handled the variation successfully” and “did something novel” are now overlapping categories, and your MES has no field for that.

This is the real shift. It’s not that robots got smarter. It’s that the line between normal execution and exception handling has gotten blurry, and MES was built on that line being sharp.

What MES actually needs to capture now

If you’re piloting a foundation-model robot cell, or evaluating one, the integration conversation needs to go well beyond the usual OPC UA tag mapping and cycle-time reporting. A few things belong in the data model that weren’t there before.

Decision provenance, not just completed steps

You need a record of what the policy actually perceived and chose, not just that a step finished. That means capturing, at minimum, which action or behavior mode the policy selected at key decision points, and ideally a lightweight trace of the inputs that drove it — the vision frame, the force reading, the part pose estimate. You’re not trying to log every inference cycle; that’s a firehose nobody can review. You’re trying to log the decision points that matter to quality and traceability, the same way you’d log a manual operator’s judgment call on a paper traveler, except now it’s machine-generated and it needs to be queryable later.

Confidence and uncertainty, as a first-class quality signal

Foundation models typically produce some notion of confidence or an uncertainty estimate alongside their action output, even if it’s not perfectly calibrated. That number is quality data. A part built by a policy that was operating at low confidence for a critical operation is not the same quality record as one built at high confidence, even if both passed inspection. Right now almost no MES schema has a place to put that. It needs one — as a genealogy attribute tied to the unit serial number, not buried in a robot log that nobody correlates back to the part.

Exception handling for “novel,” not just “failed”

Classic exception handling assumes failure: fault, stop, alarm, andon. You need a third bucket now — the policy did something outside its typical behavior envelope but completed the task anyway. That’s not a fault in the traditional sense, but it’s also not nothing. It should route to a review queue, get flagged in genealogy, and ideally get sampled for offline evaluation, the same way you’d increase inspection frequency after a process change. If your exception-handling logic only has two states, pass and fault, you will either drown quality engineers in false alarms or, worse, let genuinely novel behavior slide through unflagged because it didn’t trip a hard limit.

Model and policy version as a genealogy attribute

This one is straightforward but easy to miss: the policy version, not just the robot’s firmware or program number, has to be part of the as-built record. Foundation models get fine-tuned, updated, and retrained. If a quality issue surfaces downstream, “which cell built this” is no longer a sufficient traceability question. “Which policy version, at what confidence, making what decisions” is the actual question, and it needs to survive in genealogy long after the pilot ends.

What’s genuinely ready, and what isn’t

The simulation-to-real pipelines behind these robots — synthetic data generation, domain randomization, sim-based reinforcement learning followed by real-world fine-tuning — are mature enough to produce robots that can handle real variation in narrow, well-instrumented tasks: kitting, bin picking, certain insertion and fastening operations. That part of the stack is not hype. It’s a legitimate answer to problems that fixed-program robotics has always struggled with.

What’s not ready is the data infrastructure to make that adaptability auditable at the plant level. There’s no widely adopted standard yet for exposing policy confidence, decision traces, or behavior-envelope deviations in a form MES and quality systems can consume. IEC 62443 gives you a security framing, ISA-95 gives you a functional hierarchy, but neither was written with a non-deterministic actor in mind. Vendors are, understandably, focused on getting the robot to work at all before they build out the observability layer plant IT actually needs.

What to actually do about it

If a physical AI pilot is coming to your line in the next cycle, don’t wait for the vendor to hand you a clean data contract — negotiate one. Ask specifically what confidence or uncertainty signal the policy exposes, and whether it can be tagged to individual units, not just aggregated into a dashboard. Ask what “novel behavior” looks like from the vendor’s side and whether it’s exposed as an event at all. Build your exception queue with a third state before you go live, not after the first quality escape forces the conversation. And treat policy version like you’d treat a firmware revision on a safety-rated controller: something that gets logged, reviewed, and never silently pushed to a production cell without a change record.

None of this requires you to become a machine learning engineer. It requires you to recognize that the “operator” in that cell is now making judgment calls, and your traceability system needs to treat it the way it’s always treated judgment — as something worth recording, not just something worth trusting.


This article was written with the assistance of artificial intelligence. While we aim for accuracy, the information may be incomplete, out of date, or incorrect, and should be independently verified before you rely on it for any decision. It is provided for general information only and does not constitute professional advice.

Related posts